Journal of Clinical Virology 34 (2005) 22-25 


JOURNAL OF 


LN Ci Ak 


VIROLOGY 


www.clsevier.com/locate/jcv 


First external quality assurance of antibody diagnostic 
for SARS-new coronavirus 


Matthias Niedrig® *, Katrin Leitmeyer® ©, Wilina Lim°, Malik Peiris “, 
John S. Mackenzie °°, Maria Zambon! 


* Robert Koch-Institute, Nordufer 20, 13353 Berlin, Germany 
> World Health Organization, Geneva, Switzerland 


© Government Virus Unit Hong Kong, Hong Kong, China 


d Department of Microbiology, The University of Hong Kong, Hong Kong, China 


© Australian Biosecurity Cooperative Research Centre, Curtin University of Technology, Perth, Australia 
! Health Protection Agency, London, UK 


Received 23 December 2004; received in revised form 17 January 2005; accepted 19 January 2005 


Abstract 


To confirm an infection with the new coronavirus (SARS-CoV) causing the severe acute respiratory syndrome (SARS) diagnostic assays for 
detection of SARS-CoV specific antibody are necessary. To evaluate the diagnostic performance of laboratories an external quality assurance 
(EQA) study was performed in 2004. Participating laboratories (9/20) correctly detected anti-SARS antibodies in serum samples without false 
positive results in an immunofluorescence assay. In contrast, only 4/13 laboratories detected most of the anti-SARS antibody positive samples 
without false positive results using enzyme immunoassays (EIA) and/or immunoblot. The overall results clearly demonstrate that serological 
diagnosis of SARS-CoV remains at an early stage of development, with further technical improvements required, particularly with respect to 


the use of SARS specific EIAs. 
© 2005 Elsevier B.V. All rights reserved. 
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Diagnosis of severe acute respiratory syndrome (SARS) 
caused by a new coronavirus (SARS-CoV) is of major 
importance for assisting the control of any future SARS epi- 
demic (Drosten et al., 2003; Ksiazek et al., 2003; Peiris et al., 
2003a,b; Kuiken et al., 2003). The World Health Organiza- 
tion (WHO) helps laboratories all over the world to develop 
laboratory capability and a global reference network which 
was initiated and coordinated by WHO has assisted in the 
provision and evaluation of diagnostic tools for detection of 
SARS-CoV (WHO, 2004a,b). Since the initial epidemic that 
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started in November 2002 in south-east China and spread 
globally in 2003, at least three separate laboratory infec- 
tion incidents have occurred with some secondary cases, 
demonstrating the need for constant vigilance and reliable 
diagnostic tools (WHO, 2004c; Heymann et al., 2004). 

For the acute phase of SARS, RT-PCR methods pioneered 
by several laboratories, are the fastest and most sensitive 
method for detection of the viral genome, and appropriate 
for use in diagnosis in the first few days after illness onset. 
Significant variation is found in the virus load in different 
patients and clinical samples, which may relate to the sever- 
ity of the disease, timing and quality of sampling (Peiris et 
al., 2003a,b). As a consequence, with current knowledge, 
a negative RI-PCR result cannot be interpreted to exclude 
SARS-CoV infection. The detection of antibodies against 
SARS-CoV is therefore currently a gold standard for con- 
firmation of SARS infection (Wu et al., 2004). Serological 
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assays based on fixed infected cells using immunofluores- 
cence assays (IFA) or SARS-CoV virus specific enzyme im- 
munoassays (EIA) have been developed in many different 
laboratories worldwide. 

One way to assess preparedness of the diagnostic 
laboratories globally is through the conduct of an external 
quality assurance (EQA) program providing characterized 
serum specimens containing SARS-CoV specific antibody. 
In collaboration with the WHO, we distributed a panel of 15 
samples consisting of anti-SARS-CoV positive and negative 
human sera (WHO, 2004a). The serum material obtained 
from SARS infected patients was provided from various 
countries where SARS-CoV cases occurred. Sera were 
checked for SARS-CoV infectivity by inoculation in Vero 
cell cultures (three passages) and by RT-PCR. The sera were 
diluted in human freshly frozen plasma negative for HIV-1, 
HBV, HCV and heated to 56 °C for 1 h before lyophilization, 
for easy distribution and were tested again by two reference 
laboratories for specific activity and unspecific binding. 

To analyze the sensitivity of different assays, the EQA 
panel consisted of sequential serum dilutions, resulting in 
samples with low and high antibody titers against SARS-CoV 
as well as definite SARS-CoV negative samples. The positive 
sera were from patients from China, UK and Germany with a 
clear clinical diagnosis of SARS. All positive sera were con- 
scientiously analyzed by expert laboratories from China and 
Germany for their anti-SARS reactivity by IFA, EIA and in 
neutralization tests. The negative samples consist of human 
plasma collected before the SARS outbreak in Germany and 
were not found reactive with any of the SARS specific diag- 
nostic assays. For analysis of the specificity, two sera with 
known nonspecific reactivity against cell and mitochondrial 
structures were included. 

Here we present the results of the first EQA study 
on SARS-CoV serological diagnosis. The study was in- 
tended as a diagnostic proficiency test, including certifica- 
tion and publication of anonymized results. Thirty labora- 
tories from 18 countries (15 European/Middle Eastern, 11 
Australian/Asian/Oceanian, 3 American, 1 African) partici- 
pated (see later). The participating institutes included mem- 
bers of the international WHO SARS reference and verifi- 
cation laboratory network, as well as national and regional 
SARS reference laboratories and three commercial laborato- 
ries (WHO, 2004b). 

Each participant received a coded panel of 15 samples 
distributed by normal mail. Samples were re-suspended in 
100 jl distilled water before use. The participants were asked 
to analyze the material with the diagnostic assays in use for 
SARS-CoV diagnosis. Information on the type/format of the 
test methods used (IFA, EIA, immunoblot, neutralization as- 
say) and whether it was an in-house assay or a commercial 
kit, was also collected. 

The following two criteria were chosen as minimum re- 
quirements for overall proficiency: (1) Laboratories had to 
detect four of the five positive samples correctly (80%); (11) 
no false positive results were allowed. Indeterminate results 


in positive or negative samples were not considered for eval- 
uation. 

About 52% (13/25) of the laboratories passed the 
minimum requirements for successful performance 
(Tables 1 and 2). Three laboratories were not considered for 
evaluation because of failure to process all EQA samples. 
The failure of the other 48% of the laboratories to achieve 
minimum proficiency standards was due to lack of sensi- 
tivity, and/or to false positive results. From 20 laboratories 
using IFA, 15 out of 17 (88%) detected the high positive sera 
(#17, #20) satisfactorily. The performance was worse for 
low titer sera (#11, #23, #25) where only 9 out of 17 (53%) 
detected SARS antibodies. Eight of the 20 laboratories had 
general problems regarding specificity, detecting negative 
samples as positive in IF assays, possibly due to inadequate 
internal positive and negative controls. It is notable that 8/11 
laboratories with a good performance used a commercial 
assay, compared with 4/10 with a poor performance. 

Six out of 10 laboratories (60%) performing EIA detected 
the high positive sera (#17, #20) satisfactorily, whereas only 
one laboratory (10%) detected the low positive serum (#23), 
in comparison to IFA where 55% of laboratories detected this 
sample. One laboratory using an EJA based on SARS specific 
peptides showed high reactivity with all samples (Table 2). 
A similar result was obtained with other peptide based EIAs 
for SARS evaluated with these samples. This could be only 
avoided with real serum specimens (data not shown). 

In general the results of the laboratories using more than 
one diagnostic assay were better than of the other laborato- 
ries. Five of the 23 laboratories using IFA assay for SARS 
diagnostic also performed an EIA for confirmation of the re- 
sults. Four of five laboratories with the best performance used 
commercial assays or more than one in-house assay. 

In conclusion, the results of this first EQA study on SARS- 
CoV antibody diagnosis indicates the need for performance 
improvement in a significant number of laboratories under- 
taking serological diagnosis of SARS-CoV. Since only few 
laboratories have class 3 safety laboratory facilities and the 
experience for developing and evaluating in-house assays, 
commercial assays may be more suitable for SARS serology 
for laboratories with alow volume of testing. These results are 
also likely to be the result of the fact that limited international 
work had been performed or published on the technical de- 
velopment of the assays in use during the time this EQA was 
performed. A follow up EQA in 12-18 months time would 
be expected to show substantial improvement as laboratories 
have a chance to develop both the performance of in-house 
assays and refine testing strategies. 

Only a few laboratories were able to develop their own 
serological assays, partly because of the constraints imposed 
by the laboratory handling of SARS-CoV virus, in contrast 
to the ease with which RT-PCR protocols were disseminated 
and implemented. Also a variety of different assay formats 
(e.g. IFA, EIA, IB) and antigen sources (e.g. purified 
virus, recombinant SARS protein, SARS specific peptides), 
contributes to the variability in performance. In this regard, 


24 M. Niedrig et al. / Journal of Clinical Virology 34 (2005) 22-25 

Table 1 

SARS-CoV external quality assurance—results immunofluorescence assay 

Lab. Sample number Correct results 
number #17 pos. #20pos. #11 pos. #23pos. #25pos. #8neg. #5neg. #19neg. #2neg. #6neg* #12neg> m% 
11b° + + + + + = = a — — = 100 
12° + + + + + — = a a — (+/—) 91 
2 + + + + (—) — — — — — — 91 
20° + + + (+/—) (+/—) — a — — = — 91 
14° + + + + (—) = 5 a a — a 91 
5b + + + (—) + — — — — — — 91 
19 + + + + (—) — — — — — — 91 
23° + + (—) + + — — — — — — Ol 
Ab‘ + + (+/—) (—) + — — — — - — 86 
30b* + + (—) (-) + — — — — — — 82 
22° + + (—) + + — = = = a (+) 82 
9b° + (-) (-) + (-) ~ ~ ~ - - - 7B 
27° + + (-) (-) (-) - - - - - - 73 
28 (—) + + (—) + (+) — — — — — ie 
8 + + + + (—) a — — — (+) (+) qs 
7 + + + + + — — (+) — (+) (+) 73 
pa (+/—) (+/—) (+/—) (+/—) (-) — — — — (+) — 7) 
15 (-) (-) (-) (-) (-) - - - - (+) - 45 
24 + + + + + (+/—) (+/—) (+) (+/—) (+/—) (+) 45 
21° (H7-) -)(-) (H/-) IH) = (H/-) (+) ~ (H-) (4) 36 
I'7b* + + n.d. (+/—) + n.d. n.d. n.d. n.d. n.d. n.d. 

6 n.d. n.d. (—) n.d. n.d. — — n.d. — — — 

1 n.d. n.d. (—) n.d. n.d. — — n.d. — — — 

Pos.: serum with antibodies directed against SARS-CoV; neg.: serum without antibodies against SARS-CoV; +: correct positive result; —: negative result; 


(+/—): indefinite; (+): false positive result; (—): false negative result; n.d.: not done. 


4 Serum with antibodies reactive with cell core structure. 
> Serum with antibodies reactive with mitochondria. 


© Use of commercial assays. Except for 9b and 17b only the IFA from EUROIMMUN was used. 


the strategy of testing with peptide based assays as a single 
test should be avoided, since it appears from the laboratories 
using this approach there is a lack of specificity. Diagnostic 
algorithms for serological diagnosis of SARS _ require 
refining. 


The approach of sharing protocols and resources for 
serology testing should be further promoted to guarantee 
dissemination of high quality assays to ensure the timely 
serological confirmation of suspected SARS cases. In the 
meantime few commercial assays, most of them EIAs are 


Table 2 

SARS-CoV external quality assurance—results EIA 

Lab. Sample number Correct results 
number #17 pos. #20pos. #11 pos. #23pos. #25pos. #8neg. #5neg. #19neg. #2neg. #6neg* #12neg> m% 

lat + + (+/—) + + a — — — a a 95 

Aa® + + + (—) + a a — a a a ot 

184 + + + (—) (+/—) _ — — — _ _ 86 

9a° + + (—) (+/—) + = = — — = = 86 

10° + (+/—) (+/—) (+/—) (+/—) — ae = a — as 82 

30a° + + + (—) + (+/—) — — (+/—) — — 73 

294 + + + + + (+) (+) (+) (+/—) — (+/—) 73 

16° (-) + (-) (-) (-) - - - - - ~ 64 

26 + + + + + (+) = (+/—) = (+) (+) ae, 

Sa (-) (-) (-) (-) (-) - - - ~ - - 55 

13 + + n.d. (—) + n.d. n.d. — n.d. n.d. n.d. 

17a° + + n.d. + + n.d. n.d. — n.d. n.d. n.d. 

2 + + + + + n.d. (+) (+) (+) (+) (+) 

Pos.: serum with antibodies directed against SARS-CoV; neg.: serum without antibodies against SARS-CoV; +: correct positive result; —: negative result; 


(+/—): indefinite; (+): false positive result; (—): false negative result; n.d.: not done. 


4 Serum with antibodies reactive with cell core structure. 
> Serum with antibodies reactive with mitochondria. 

© Use of commercial assays. 

d Use of in-house immunoblot assay. 
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distributed or under development (Chang et al., 2004; Guo 
et al., 2004; He et al., 2004). The evaluation of these assays 
based on the accessibility of well-characterized SARS-CoV 
positive sera is still a major problem for all laboratories and 
companies working on this task. 

In the WHO recommended diagnostic strategy, sero- 
conversion is required to diagnose acute infection. For 
establishing serostatus, while sensitivity is very impor- 
tant for screening, due to the very low seroprevalence 
worldwide, sequential testing using various different anti- 
gens/methodologies is required to get acceptable positive 
predictive value. The present EQA could not take into ac- 
count this objective of testing or different testing strategies; 
therefore it is difficult to assess the proficiency of each 
participant for acute diagnosis or serostatus determination. 

When testing for seroconversion, using paired sera, the 
sensitivity of the assays might be less important than a high 
specificity to avoid false positive results. Nevertheless highly 
sensitive and good specific serology assays will help to 
improve the diagnostic performance of the laboratories with 
a clear benefit for the patient. The EQA performed for SARS 
antibody diagnostic proved a clear advantage for all par- 
ticipants, either by demonstrating an adequate performance 
quality or demonstrate the need for necessary technical 
improvements or the training of the laboratory personal. 
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